Artificial Neural Networks for Forecasting Stock Price ]

Statistical arbitrage strategies have always been popular since the advent of algorithmic trading. In particular, Exchange traded fund (E.T.F.) arbitrage has attracted much attention. Trading houses have tried to replicate ETF arbitrage to other stocks. Thus, the objective is to be able to develop a long term pricing relationship between stocks and profit from their divergence from this relationship. In this paper, we have developed a feasible trading strategy on this concept. Artificial neural networks have been deployed to model the pricing relationship between elements in a sector. All prices have been considered at the same instant, thereby allowing us to make trading decisions in accordance with our predictions. Supervised learning algorithms have been used to train the network. INTRODUCTION Artificial Neural Networks (ANN) have been shown to be an efficient tool for non-parametric modeling of data in a variety of different contexts where the output is a non-linear function of the inputs. These include business forecasting, credit scoring, bond rating, business failure prediction, pattern recognition and image processing. Furthermore, in the field of finance and stock price prediction, some work has been undertaken to model stock prices using ANN. Yet, there doesn’t exist any feasible statistically profitable strategy that tends to employ ANN efficiently. The price of a stock is influenced by: 1. Company news – Any news pertaining to the company being studied. 2. Macro-economic news – Any news pertaining to the broader section of the stock market. Company specific news is significantly lower in frequency than macroeconomic news and thus its important point to automatically profit from macro-economic news. We would like to do this algorithmically owing to the large volume of information involved. Stock price prediction, market sentiment analysis and arbitrage are the commonly applied concepts in development of a trading strategy. Artificial Neural Networks for Forecasting Stock Price 2008 Page 3 of 15 The major problem with stock price prediction is accounting for the vast volume of information affecting stock prices. As George Soros put it in 1987: One of the major problems faced in modeling financial market movements is the fact that information comes in from a very large number of sources. Keeping this in mind, this study discounts all non-specific news by its impact on other stocks in the same sector. It is assumed that in the absence of company specific news, there exists pricing relationships between elements in a sector. Further stock prices conform to a pricing relationship that can be ‘learnt’ from its historical data. Thus, basically the neural network will model a response relationship amongst elements from a sector. We have studied two sectors, the Indian Construction sector and the U.S. technology sector. Data from different time periods and different nature has been used in the study. While intraday data for October 1, 2004 has been used in training and testing the neural network associated with Yahoo (U.S. technology sector), end of day data from July 24 ‘07– 4 November ’08 has been employed for Unitech (Indian construction company). ARTIFICIAL NEURAL NETWORKS In recent years some work has been reported on the use of ANNs for analysis of financial markets. In what follows we provide a very brief introduction to ANNs following Stern (1996). The three essential features of ANN are basic computing elements referred to as neurons, the network architecture describing the connections between the neurons and the training algorithm used to find values of the network parameters for performing a particular task. Each neuron performs a simple calculation, a scalar function of a scalar input. Suppose we label the neurons with positive integers and denote the input to the k neuron as ik and the output from the k neuron as ok. Then ok = fk(ik) where f(.) is a specified function that is typically monotone but otherwise arbitrary. The neurons are connected to each other in the sense that the output from one unit can serve as part of the input to another. There is a weight associated with each connection; the weight from unit j to unit k is denoted as wjk. Let N(k) denote the set of units that are connected to unit k. The input to unit k is then ik = ∑jєN(k)wjkoj. Network architecture refers to the organisation of the neurons and the types of connections permitted. In the multilayer feed forward network, the type used in this article, neurons are organised in a series of layers. Information flows only in one direction; units receive information only from units in higher layers of the network. Artificial Neural Networks for Forecasting Stock Price 2008 Page 4 of 15 ANN can be ‘trained’ to perform a specific task by adjusting these weights. The weights are continually adjusted by comparing the output of the network with the target until the output of the network ‘matches’ the target in the sense that the error function measuring the difference between the target and the output is minimised. Many pairs of input and output are used to train the network and this mode of adjustment is called ‘supervised’ learning. There are several approaches for minimisation of the error function including standard optimisation techniques like conjugate gradient and Newton-type methods. However, the most popular algorithm in this context is the backpropagation algorithm of Rumelhart et al. (1986). The algorithm is extremely simple to program but tends to converge slowly. Training of a network can take two forms: ‘incremental’ or ‘batch’. In case of incremental training, individual pairs are fed one at a time to the network. Output is compared with the target for each input and adjustments of the weights are made using a training algorithm, most often the backpropagation algorithm. The next pair is then fed to the network after making the adjustment for the previous pair and the same process is repeated. Incremental training is sometimes referred to as ‘on line’ or ‘adaptive’ training. In case of batch training, all the pairs of input and output are fed to the network at the initial stage and the weights are adjusted. In case of unsupervised learning, outputs for a given set of inputs are not available. Unsupervised learning is mostly used for networks that perform clustering and pattern recognition. Cheng and Titterington (1994) provide a comprehensive review of ANNs from a statistical perspective. A well-trained ANN can exploit the underlying non-linear relationships that drive security prices. Further, the networks can be retrained using newer data. Thus, a network can adapt to new information as it comes. This makes ANNs a very promising tool for security price modelling. The vast increase in computing power has made modelling of complex systems relatively easy and less time consuming than it was in the past. LITERATURE REVIEW ANNs are being used in various fields of application including business forecasting, credit scoring, bond rating, business failure prediction, medicine, pattern recognition, image processing, speech processing, computer vision and control systems. In the context of financial forecasting, Kuan and Liu (1995) discuss forecasting of foreign exchange rates using ANNs. They show that a properly designed ANN has lower out-of-sample mean squared prediction error relative to the random walk model. Jasic and Wood (2004) discuss the profitability of trading signals generated from the out-of-sample short-term predictions for daily returns of S&P; 500, DAX, TOPIX and FTSE stock market indices evaluated over the period 1965–99. The out-of sample prediction performance of neural networks is compared against a benchmark linear autoregressive model. They find that the buy and sell signals derived from neural network predictions are significantly different from unconditional one-day mean return and are likely to Artificial Neural Networks for Forecasting Stock Price 2008 Page 5 of 15 provide significant net profits for reasonable decision rules and transaction cost assumptions. Cao et al. (2005) provide a comparison between the Fama and French model and the ANN model in the context of prediction of the Chinese stock market. They report that ANNs outperform the linear models from financial forecasting literature in terms of its predictive power. Tkacz (2001) provides an interesting study regarding the use of leading indicator neural network models for forecasting Canadian GDP growth. It is reported that the neural network models yield statistically lower forecast errors for the year-over year growth rate of real GDP relative to linear and univariate models. Huang et al. (2004) report a comparative study of application of Support Vector Machines (SVM) and Backpropagation Neural Networks (BNN) for an analysis of corporate credit ratings. They report that the performances of SVM and BNN in this problem were comparable and both these models achieved about 80 per cent prediction accuracy. Pendharkar (2005) discusses the application of ANNs for the bankruptcy prediction problem. It is reported that ANNs perform better than the statistical discriminant analysis both for training and hold-out samples. HYPOTHESIS We hypothesize that stock prices move in a complex fashion, yet this pattern can be ‘learnt’ over the short term using technical tools like ANN. Learnt on historical data, and in between periods of company specific news, ANN’s can be deployed to produce a feasible trading strategy. Further, a strategy mirroring index arbitrage can be developed based on the ANN, which will statistically yield profits. TIME SERIES ECONOMETRICS VERSUS NEURAL NETWORKS Time series forecasting analyzes past data and projects estimates of future data values. Basically, this method attempts to model a nonlinear function by a recurrence relation derived from past values. The recurrence relation can then be used to predict new values in the time series, which hopefully will be good approximations of the actual values. There are two basic types of time series forecasting: univariate and multivariate. Univariate models, like Box Jenkins, contain only one variable in the recurrence equation. Box Jenkins is a complicated process of fitting data to appropriate model parameters. The equations used in the model contain past values of moving averages and prices. Box Jenkins is good for short term forecasting but requires a lot of data, and it is a complicated process to determine the appropriate model equations and parameters. Artificial Neural Networks for Forecasting Stock Price 2008 Page 6 of 15 Multivariate models are univariate models expanded to "discover casual factors that affect the behavior of the data." As the name suggests, these models contain more than one variable in their equations. A neural network is able to work parallel with input variables and consequently handle large sets of data swiftly. The principal strength with the network is its ability to find patterns and irregularities as well as detecting multidimensional nonlinear connections in data. The latter quality is extremely useful for modeling dynamical systems, e.g. the stock market. Neural networks are used to predict stock market prices because they are able to learn nonlinear mappings between inputs and outputs. Economists have used time series econometrics to estimate relationship between stock prices but to little practical success. Mainly as a consequence of close to random walk behavior of the stock time series, nonparametric techniques have failed and in this paper we have resorted to neural networks. DATA SOURCES AND DATA ADJUSTMENT Unitech(Indian Construction and Contracting companies) End of day data has been procured from finance.yahoo.com. End of day data has been recorded for Indian construction companies that have a net worth in excess of 2000 Cr as of Nov’ 04 2008. Thus, the following have been recorded. 1. AKRUTI – 7 Feb ‘07 to 4 Nov ’08 2. ANSAL INFRA – 27 Jan ‘06 to 4 Nov ’08 3. DLF – 5 July ’07 to 4 Nov ’08 4. HDIL – 24 July ’07 to 4 Nov ’08 5. IBRINFRA – 23 March ’07 to 4 Nov’ 2008 6. IRB – 25 Feb ’08 to 4 Nov ’08 7. IVRCLINFRA – 3 Nov ’03 to 4 Nov ’08 8. JPASSOCIAT14 June ’04 to 4 Nov ‘08 9. MAYTASINFR25 Oct ’07 to 4 Nov ‘08 Since we need to develop complete data vectors we need to take the intersection of all these dates. Thus, yielding data from 25 Feb ’08 to 4 Nov’ 2008. Even procuring two data vectors per day (start of day and end of day data), this yields insufficient amount of data for training and testing the neural network. Thus, for sake of convenience and without loss of generality, we have removed IRB and MAYTASINFR. Thus, the input to the ANN are vectors generated from the 7 remaining companies. Further still, corresponding to each day, 2 data vectors have been generated, corresponding to the starting price and the closing price of the day since these present two feasible prices. Thus a total of 640 data vectors have been prepared for our study. Symbol Date Prev Close Open Price High Price Low Price Last Price Close Price Artificial Neural Networks for Forecasting Stock Price 2008 Page 7 of 15 MAYTASINFR 25-Oct-07 370 515.75 642 515 630 613.35 MAYTASINFR 26-Oct-07 613.35 635 716 625 633.35 640.2 MAYTASINFR 29-Oct-07 640.2 640.2 658.85 591.1 613 610.15 MAYTASINFR 30-Oct-07 610.15 620 624.95 597 618.25 615.7 MAYTASINFR 31-Oct-07 615.7 560 738.85 560 738.85 738.85 MAYTASINFR 1-Nov-07 738.85 752.3 818.8 676.25 710 700.35 MAYTASINFR 2-Nov-07 700.35 675 740 671.35 707 708.55 MAYTASINFR 5-Nov-07 708.55 715 745 690 717.1 721.85 YAHOO(U.S. Technology Sector) Intra-day data has been procured for 1 Oct’ 04 for YHOO and other U.S. technology stocks. A short subsection is as below: AMZN,20041001,8:00:01,40.7000 EBAY,20041001,8:01:01,92.0500 EBAY,20041001,8:01:03,91.9900 AMZN,20041001,8:02:00,40.9050 EBAY,20041001,8:02:01,91.9000 YHOO,20041001,8:03:42,34.2400 The format is symbol,date,timestamp,price. The data has intra-day transaction for 8 stocks besides YHOO. A total of 155,052 intra-day trades took place. We have constructed the feasible price data vectors as follow: 1. Fill in feasible data vectors with recent prices till all 8 stocks have appeared atleast once. 2. Now for every new trade, change the current price at the stock traded position to generate a new feasible price vector. 3. In this fashion, read the complete file to generate all feasible price vectors. These will serve as inputs to our ANN. Care must be taken, when working with intra-day data to discourage use of data whereby any company specific news may have been announced during the trading period. DATA VECTOR AT TIME T Special care has been taken to choose the input variables very carefully. We want to mirror a deployable trading strategy and thus, it would be unwise to include any lagged variables. Thus, although the best determinant of the time of YHOO at time ‘t+1’ is probably price of YHOO at time ‘t’, yet YHOO(-1) has not been included. The idea behind such a strategy choice is that we cannot trade YHOO(-1). Given a mispricing whereby we have lagged variable involved, we Artificial Neural Networks for Forecasting Stock Price 2008 Page 8 of 15 cannot conduct a series of trades that would mirror a hedged position. Owing to similar reason we have not included popular indicators in the equations like Moving Average and Oscillators. Thus, in our choice of variables we have included only real time variables. Further the reasons for choosing companies from the same sector are obvious. We have assumed that the sector discounts all macro-economic news in a similar and understandable fashion. And we have chosen the big companies from the sector since geographic biases can be assumed annulled. TRADING STRATEGY The trading strategy employed will mirror plain index arbitrage strategy. Any divergence from the pricing relationship (in this case devised from the ANN) will be traded upon. For sake of convenience we will assume no commissions involved. Thus, at every given point in time we will compute the difference between the true price of the stock and its ANN generated value. Further still we will approximate the composition of each underlying (the composition of inputs that make the output) linearly using a Taylor series expansion to one term. If the underlying are ‘relatively expensive’ (ANN price > True price) then we will short the underlying in the ratio of their composition (as per the linear approximation to the pricing relationship of the ANN at that point) and long the stock. On the other hand, if the underlying are ‘relatively cheaper’ then we will long the underlying and short the stock. We will continue this process till the time there is greater divergence from the relationship. When the divergence reduces to zero we close our trades. This strategy is not a ‘pure arbitrage’ but a case of statistical arbitrage due to two main reasons: 1. The composition and linear approximations are done locally. Thus, we don’t exactly hedge our risks at all given points in time. 2. We make the trading decisions at time ‘t’, yet the trade takes place at time ‘t+1’. Thus, there may be an overlap of the price curves and we lose significantly. 3. Furthermore, we have made a huge assumption of conforming to the pricing relationship established by the ANN. If this hypothesis is violated we stand to lose. METHODOLOGY A primary drawback with ANN is the fact that there is no accepted procedure for the development of an accepted architecture; meaning the transfer functions to be used, the number of neurons per layer and the number of hidden layers. Another important point to be noted is that since the starting weights in the backpropogation algorithm are randomly set, running the ‘learning’ phase twice on the same dataset might yield two different results. Artificial Neural Networks for Forecasting Stock Price 2008 Page 9 of 15 In the problem deployed we have used MATLAB Neural Network Toolbox for designing the neural network. Further still Levenberg-Marquart algorithm has been used in the training period. We have tried to minimize the mean square error of the fitted data. Owing to the iterative procedure associated with development of the architecture, we have run 10 iterations, computing the mean square error on each instance using an epoch setting of 50. We have tested with 2 and 3 hidden layers and using between 5 and 8 neuron per layers, computing the mean square error per architecture. Keeping in mind the problem of over fitting the model, we have manually chosen the best possible network. Technology sector of U.S.A. In our model associated with the prediction of the correct price of YHOO using intra-day data we have deployed a 3 hidden layer ANN. The transfer functions employed are ‘tansig’ in the first, with 7 neurons, ‘tansig’ in the second with 5 neurons and further the last having ‘logsig’ transfer function, with another 5 neuron. Using this model, we have trained our network on 90% of the data and tested on the remaining 10% data. For purposes of testing, keeping the issues relating to deployment of the network in mind, we have taken the last 10% of the data set. This is in alliance with any deployment issues that might arise. Data division between the training and the validation has been made entirely random. Construction and Contracting sector in India In our model associated with the prediction of the correct price of Unitech using end-of-day data we have deployed a 2 hidden layer ANN. The transfer functions employed are ‘tansig’ in the first, with 7 neurons and ‘purelin’ in the second with 5 neurons. Owing to a significantly smaller data set comprising only of 640 data vectors, in a larger model the problems associated with over fitting are more prominent. Using this model, we have trained our network on 90% of the data and tested on the remaining 10% data. For purposes of testing, keeping the issues relating to deployment of the network in mind, we have taken the last 10% of the data set. This is in alliance with any deployment issues that might arise. Data division between the training and the validation has been made entirely random. All matlab programs, data sets and entire project details maybe downloaded from http://home.iitk.ac.in/~ramnik/eco311.html RESULTS The price movement of Unitech from July 24 ’07 – Nov 4 ’08 on the National Stock Exchange of India was: Artificial Neural Networks for Forecasting Stock Price 2008 Page 10 of 15 Similarly, YHOO followed the trajectory as shown below, on 1 Oct ’04. YHOO Deploying ANN’s to YHOO price movement gave good results. The ANN learnt the data as follows: Artificial Neural Networks for Forecasting Stock Price 2008 Page 11 of 15 The data fit is quite good. We have chosen 90% of the data for the training period. On the testing data set, the performance of the ANN was as follows: As has been shown in the diagram, the blue curve and the red curve do overlap (in conjunction with our hypothesis). Also the data has been fit very well (within 1 percent of the true value). The places where the curves diverge can be potentially used to generate profits. The trades deployed are as shown. The profit curve from the trades is simulated as follows: Artificial Neural Networks for Forecasting Stock Price 2008 Page 12 of 15 The small spikes in the bottom (the green ellipse) are the trading profits per unit trade. The summation of these unit trade profits generates the total profit curve. The diagram from trade profit per unit trade has been replicated below: The point to note is that trades have been simulated by including commissions and under the assumption that trade at time instant ‘t’ can be completed at the same price. Trades have been Artificial Neural Networks for Forecasting Stock Price 2008 Page 13 of 15 simulated on the testing period only. And the Total Profit is also shown: Unitech Using 90% of the data for learning and remaining for testing, under the architecture as defined above, the plot of the training data versus true data is: It must be noted that there is high volatility in the price of Unitech. The price range is also high, ranging from 500 to close to 20. Further, the markets have been choppy. Yet, the non linear Artificial Neural Networks for Forecasting Stock Price 2008 Page 14 of 15 relationship is learnt to high degrees of accuracy by the neural network. The performance of the ANN on the testing data is as follows: As can be seen from the two curves, the ANN does not trace the true data to even a reasonable degree of accuracy. As has been labeled on the diagram, the estimated price of Unitech is off its true value by 700%. Furthermore, the prices never converge (in the time period shown). Thus replicating index arbitrage on this data would not be feasible, and the ANN model would fail. CONCLUSION ANN forms a useful tool in predicting price movement of a particular stock. In the short term, the pricing relationship between the elements of a sector holds firmly. An ANN can learn this pricing relationship to high degree of accuracy and deployed to generate profits. With sufficiently large amounts of data, preferably in times of low volatility and over a short time period, ANN can be deployed to replicate index type statistical arbitrage on any stock. As can be concluded from the performance of the trading strategy on YHOO; when trained on large intra-day data, ANN’s tend to perform well. ANN’s predicted price to the accuracy of 1% and the hypothesis of convergence to the relationship was frequently justified. Furthermore, we were successfully able to generate a reasonable profit (in the simulated real time testing environment using 10% testing data vectors) Also from the performance of ANN on Unitech we can conclude that in presence of data spreading over many days, and markets being choppy over the time period, ANN performs well below desired level of precision. Using end-of-data is not a feasible. Artificial Neural Networks for Forecasting Stock Price 2008 Page 15 of 15Further testing with intra-day real time data would be important to certify the profitability ofthis strategy. Work may also be done using ANN with inputs spread over the entire marketinstead of concentrating on a particular sector. Finally, work may be carried out on the optimaldesign of the network. REFERENCESCao, Q., K.B. Leggio and M.J. Schniederjans (2005), ‘A Comparison Between Fama andFrench’s Model and Artificial Neural Networks in Predicting the Chinese Stock Market’,Computers and Operations Research, 32: 2499–512. Cheng, B. and D.M. Titterington (1994), ‘Neural Networks: A Review from a StatisticalPerspective’, Statistical Science, 9(1): 2–30. Huang, Z., H. Chen, C-J Hsu, W-H Chen and S. Wu (2004), ‘Credit Rating Analysis withSupport Vector Machines and Neural Networks: A Market Comparative Study’, DecisionSupport Systems, 37(4): 543–58. Jasic, T. and D. Wood (2004), ‘The Profitability of Daily Stock Market Indices Trades Basedon Neural Network Predictions: Case Study for the S&P; 500, the DAX, the TOPIX andthe FTSE in the Period 1965–1999’, Applied Financial Economics, 14(4): 285–97. Kuan, C-M and T. Liu (1995), ‘Forecasting Exchange Rates Using Feedforward and RecurrentNeural Networks’, Journal of Applied Econometrics, 10(4): 347–64. Pendharkar, P.C. (2005), ‘A Threshold-varying Artificial Neural Network Approach forClassificationand its Application to Bankruptcy Prediction Problem’, Computers and OperationsResearch, 32(10): 2561–82. Soros, G. (1987), The Alchemy of Finance: Reading the Mind of the Market. New York: JohnWiley & Sons, Inc. Stern, H.S. (1996), ‘Neural Networks in Applied Statistics’, Technometrics, 38(3): 205–14. Tkacz, G. (2001), ‘Neural Network Forecasting of Canadian GDP Growth’, InternationalJournal of Forecasting, 17(1): 57–69.